Python: fix FIDES session isolation and runtime integration gaps by lerelerele · Pull Request #7528 · microsoft/agent-framework

Oscar Lobato Ruiz (lerelerele) · 2026-08-05T11:33:14Z

• Closes #7455

This PR implements the A1 - C3 changes described in #7455.

The main architectural change is to make FIDES runtime state conversation-scoped. Context labels, audit records, pending approvals, counters, and variable stores are now associated with AgentSession.state instead of shared middleware or agent instances. This prevents state leaking between concurrent conversations that reuse the same agent or middleware.

What changed

A1: Store context labels, audit logs, pending approvals, and turn/call counters in AgentSession.state; variable stores are isolated per session.
A2: A blocked call now returns a correlated function_result instead of terminating the loop with empty output, allowing the model to explain the refusal or choose another action. Approval requests still pause for the user.
A3: Audit records now carry a per-run turn and per-call call_index.
B1: Add tool_labels configuration for externally-created tools, including harness and MCP tools.
B2: Preserve approval-request additional_properties when reconstructing function calls.
B3: Prevent standing approval rules and automatic approval callbacks from approving FIDES policy violations.
B4: Add the public build_function_call_content() extension point while retaining the former private method as an alias.
B5: Add deny_untrusted_tools, with deny rules taking precedence over allow rules.
C1: Support enable_quarantine=False, allowing labels and policy enforcement without quarantine tooling.
C2: Bind the quarantine client to the current async context instead of a process-global slot.
C3: Apply a PUBLIC confidentiality cap to read-only MCP tools, with mark_read_tools_as_sinks as an opt-out.

The durable session.state["_fides"] schema is owned by _fides_session_state(). It creates the mapping and initializes the context label, audit log, pending approvals, and turn/call counters in one place. before_run(), the label
accessors, and the policy-enforcement accessors all route through it; their per-key initialization has been removed.

_current_middleware now uses a plain async-safe ContextVar, replacing the previous threading.local() storage. No-session fallbacks remain local, and direct middleware calls still return turn 1 on their first call.

Audit logs can be bounded with max_audit_log_entries, which defaults to 1000.

Additional API surface

This PR also includes two additions that were not explicitly listed in #7455:

max_audit_log_entries, defaulting to 1000 and accepting None for an unlimited log.
begin_turn(session), which provides the run boundary needed to populate meaningful audit turn numbers.

These additions are called out explicitly for maintainer review.

Scope

This PR is scoped to the Python core FIDES integration work described in #7455.

Tests and verification

Regression coverage was added to the existing test suites:

python/packages/core/tests/test_security.py
python/packages/core/tests/core/test_harness_tool_approval.py

The standalone regression test module was removed.

Full core test suite excluding integration tests:
pytest packages/core/tests -m "not integration" -q

FIDES kept conversation scoped security state on shared middleware instances, so a single Agent or middleware instance serving more than one conversation could leak context labels, audit records and pending approvals across conversation boundaries. A blocked tool call also terminated the invocation loop without a visible result, which could reach the user as an empty response. Move conversation scoped state into AgentSession.state and address the A1 to C3 items from microsoft#7455: - A1: context label, audit log, pending approvals and counters live in AgentSession.state; variable stores are keyed per session. - A2: a blocked call returns a correlated function_result instead of terminating with no content, so the model can explain the refusal or choose another action. Approval requests still pause for the user. - A3: audit records carry a per run turn and a per call call_index instead of the constant -1. - B1: new tool_labels configuration for tools the application did not construct, such as harness and MCP tools. - B2: approval request additional_properties propagate into the reconstructed function call. - B3: standing approval rules and auto approval callbacks can no longer approve a FIDES policy violation. - B4: new public build_function_call_content() extension point; the former private method remains as an alias. - B5: new deny_untrusted_tools; deny takes precedence over allow. - C1: new enable_quarantine=False so labels and policy enforcement can be used without quarantine tooling. - C2: the quarantine client is bound to the current async context instead of a process global slot. - C3: MCP read only tools receive a PUBLIC confidentiality cap, since their arguments still leave the process, with a granular mark_read_tools_as_sinks opt out. Also hardens durable state: label metadata is sanitized before it reaches session storage, and the audit log is capped per session with a configurable limit. Middleware order is unchanged, and direct middleware calls made without an AgentSession keep working through explicit fallbacks. Adds 21 regression tests in tests/test_fides_7455_regressions.py.

Oscar Lobato Ruiz (lerelerele) · 2026-08-05T11:37:02Z

@microsoft-github-policy-service agree

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

python/packages/core/agent_framework/security.py:2425

PolicyEnforcementFunctionMiddleware.clear_audit_log() has the same session-resolution issue as get_audit_log(): when called without an explicit session, it uses get_current_session() which may be None after middleware execution. Fall back to the remembered active session so the public API behaves consistently.

    def clear_audit_log(self, session: Any = None) -> None:
        """Clear the audit log."""
        self._resolve_audit_log(session or get_current_session()).clear()

python/packages/core/agent_framework/security.py:2421

PolicyEnforcementFunctionMiddleware.get_audit_log() only falls back to get_current_session(), which is cleared when the label-tracking middleware exits. This means calling get_audit_log() without explicitly passing session can incorrectly return the no-session fallback log even though a session was previously active in the current async context (contrary to the module’s new “remember last session” behavior). Consider also falling back to the weakly remembered _fides_active_session.

This issue also appears on line 2423 of the same file.

            List of violation records.
        """
        return self._resolve_audit_log(session or get_current_session()).copy()

Evan Mattson (moonbox3) · 2026-08-06T06:55:42Z

It's a bit confusing why the PR body links to an issue that says that this PR doesn't solve. Can you please link to the correct issue?

- Reuse make_json_safe for ContentLabel Enum and set serialization. - Keep middleware state in plain ContextVars and route FIDES state access through shared helpers. - Move regression coverage into existing test suites. - Resolve the remembered session for audit-log accessors after run teardown.

Oscar Lobato Ruiz (lerelerele) · 2026-08-06T11:13:32Z

You’re right. I updated the PR description to reference the correct issue, #7455, using Closes #7455. The unrelated #7466 reference has been removed.

I also addressed the other review suggestions:

“I don't think this net-new file is needed.”

Removed test_fides_7455_regressions.py and moved all 21 regression cases into the existing suites: 20 into test_security.py and the standing-approval case into core/test_harness_tool_approval.py.

“Please keep JSON safety behind the existing make_json_safe method…”

Removed the local _json_safe helper and extended make_json_safe() to handle Enum values and sets/frozensets. ContentLabel.to_dict() now delegates to it, leaving a single owner for JSON safety.

“Let’s look at having _current_middleware remain a plain ContextVar…”

_current_middleware is now a plain async-safe ContextVar, replacing the previous threading.local() storage. Production code uses set(), reset(), and get(), and the tests no longer rely on the .instance property.

“How about one helper owns initialization and access for session.state["_fides"]…”

_fides_session_state() is now the single owner of the durable session.state["_fides"] schema. It creates the mapping and initializes the context-label slot, audit log, pending approvals, and turn/call counters.

before_run(), the label accessors, and the policy-enforcement accessors all route through it, so their per-key initialization has been removed. No-session fallbacks remain local, and direct middleware calls still return turn 1 on
their first call.

One intentional consequence is that any FIDES session-state access materializes the complete schema, including empty audit, approval, and counter fields. This keeps the durable contract owned by one accessor.

While validating the changes, I also found and fixed a separate audit-log issue: after a run completed, get_audit_log() and clear_audit_log() could fall back to the middleware’s empty log because the active session had already been
cleared. Both accessors now resolve the active or last remembered session through _resolve_active_session(), with regression coverage in the existing TestPolicyEnforcementMiddleware suite.

Focused FIDES tests, the full core suite excluding integration tests, Ruff, and Pyright all pass.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Suppressed comments (16)

python/packages/core/tests/test_security.py:1504

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:1536

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:1592

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:1619

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:1638

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:2129

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:2145

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:2166

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:2188

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:2213

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:3100

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:4230

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/tests/test_security.py:4247

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

    @pytest.mark.asyncio

python/packages/core/agent_framework/security.py:2131

Calling the new public method directly bypasses subclasses that override the former private extension point, so the private "compatibility alias" does not actually preserve those overrides. Call the private alias here; its base implementation delegates to the public method, which supports both old and new subclasses.

            function_call=self.build_function_call_content(context),

python/packages/core/agent_framework/_tools.py:1696

This introduces a new exception to the function-loop middleware-termination contract, but the required contract and scenario-to-test matrix were not updated. python/AGENTS.md:60-67 requires every function-loop behavior/coverage change to update docs/specs/004-python-function-calling-loop.md; its current middleware-termination row still says termination always stops without another model call. Add the blocked-policy continuation scenario and its streaming/non-streaming regression coverage to that specification.

            # A blocked FIDES call is a normal tool result: the model must receive
            # the refusal and get a chance to explain or choose another action.
            # Approval requests remain terminal and pause for user input.
            is_blocked_policy = (exc.result.additional_properties or {}).get("blocked_violation") is True
            return [exc.result], not is_blocked_policy

python/packages/core/tests/test_security.py:670

Remove this decorator. Python tests use asyncio_mode = "auto", and the repository test guidance explicitly requires plain async def tests without @pytest.mark.asyncio (python/.github/skills/python-testing/SKILL.md:49-52).

This issue also appears in the following locations of the same file:

line 1504
line 1536
line 1592
line 1619
line 1638
...and 8 more

    @pytest.mark.asyncio

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

python/packages/core/agent_framework/security.py:1848

Creating a ContextVar per enforcer causes the current Context to retain every variable key it has seen, and each value here strongly retains the session’s _fides state through _FidesAuditRunContext.scope. Since before_run() always calls begin_turn(), long-lived tasks retain at least the latest conversation state for every config/enforcer created, defeating the weak-session design above. Use a module-level context variable with weak/config-scoped entries, or reset the binding at the run boundary.

        self._direct_audit_run: ContextVar[_FidesAuditRunContext | None] = ContextVar(
            f"_fides_direct_audit_run_{id(self)}", default=None
        )

python/packages/core/agent_framework/security.py:110

This new durable FIDES session-state owner is the PR’s stated architectural change, but no ADR is included. The repository convention requires significant architectural changes to be documented under docs/decisions/; please add one covering the shared _fides schema, session/child-run sharing, compatibility, and lifecycle tradeoffs.

def _fides_session_state(session: Any) -> dict[str, Any]:
    """Return the initialized ``session.state["_fides"]`` durable-state mapping.

    This is the single owner of the FIDES session-state contract. Callers that may not
    have a session must guard for ``None`` before calling.

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

python/packages/core/agent_framework/security.py:379

_storage can now be AgentSession.state["_fides"]["variables"], so storing content verbatim makes otherwise valid ContentVariableStore.store(Any, ...) calls capable of breaking durable session persistence. For example, bytes or an unregistered object are accepted here, but FileSessionStore rejects unsupported serialized state in _sessions.py:433-449. Encode/validate stored content for the durable-state contract (or define a reversible codec) and cover an actual FileSessionStore round trip.

        self._storage[var_id] = {"content": content, "label": label.to_dict()}

+            # A blocked FIDES call is a normal tool result: the model must receive
+            # the refusal and get a chance to explain or choose another action.
+            # Approval requests remain terminal and pause for user input.
+            is_blocked_policy = (exc.result.additional_properties or {}).get("blocked_violation") is True


Copilot AI lite review requested due to automatic review settings August 5, 2026 11:33

Oscar Lobato Ruiz (lerelerele) temporarily deployed to github-app-auth August 5, 2026 11:33 — with GitHub Actions Inactive

Oscar Lobato Ruiz (lerelerele) had a problem deploying to github-app-auth August 5, 2026 11:33 — with GitHub Actions Error

Oscar Lobato Ruiz (lerelerele) temporarily deployed to github-app-auth August 5, 2026 11:33 — with GitHub Actions Inactive

agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Aug 5, 2026

Oscar Lobato Ruiz (lerelerele) temporarily deployed to github-app-auth August 5, 2026 11:33 — with GitHub Actions Inactive

Copilot started reviewing on behalf of Oscar Lobato Ruiz (lerelerele) August 5, 2026 11:33 View session

Copilot AI reviewed Aug 5, 2026

Oscar Lobato Ruiz (lerelerele) requested a lite review from Copilot August 5, 2026 15:36

Copilot started reviewing on behalf of Oscar Lobato Ruiz (lerelerele) August 5, 2026 15:37 View session

Copilot AI reviewed Aug 5, 2026

View reviewed changes

Evan Mattson (moonbox3) reviewed Aug 6, 2026

View reviewed changes

Oscar Lobato Ruiz (lerelerele) temporarily deployed to github-app-auth August 6, 2026 10:52 — with GitHub Actions Inactive

Oscar Lobato Ruiz (lerelerele) temporarily deployed to github-app-auth August 6, 2026 11:02 — with GitHub Actions Inactive

Oscar Lobato Ruiz (lerelerele) added 2 commits August 7, 2026 14:44

Python: preserve upstream byte serialization support

babb0c2

Merge main into FIDES fixes

b818200

Oscar Lobato Ruiz (lerelerele) temporarily deployed to github-app-auth August 7, 2026 12:47 — with GitHub Actions Inactive

Merge branch 'microsoft:main' into codex/fides-7455-fixes

2208fc8

Oscar Lobato Ruiz (lerelerele) deployed to github-app-auth August 11, 2026 08:14 — with GitHub Actions Active

Oscar Lobato Ruiz (lerelerele) requested a lite review from Copilot August 11, 2026 09:57

Copilot AI reviewed Aug 11, 2026

Oscar Lobato Ruiz (lerelerele) requested a balanced review from Copilot August 11, 2026 10:45

Copilot started reviewing on behalf of Oscar Lobato Ruiz (lerelerele) August 11, 2026 10:45 View session

Copilot AI reviewed Aug 11, 2026

View reviewed changes

Comment thread python/packages/core/agent_framework/security.py Outdated

Comment thread python/packages/core/agent_framework/security.py Outdated

Comment thread python/packages/core/agent_framework/security.py Outdated

Python: Harden FIDES run and session isolation

8452e59

Merge remote-tracking branch 'origin/main' into codex/fides-7455-fixes

015570c

Oscar Lobato Ruiz (lerelerele) deployed to github-app-auth August 11, 2026 15:55 — with GitHub Actions Active

agent-framework-automation Bot added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label Aug 11, 2026

Oscar Lobato Ruiz (lerelerele) requested a balanced review from Copilot August 11, 2026 16:03

Copilot started reviewing on behalf of Oscar Lobato Ruiz (lerelerele) August 11, 2026 16:04 View session

Copilot AI reviewed Aug 11, 2026

View reviewed changes

Comment thread python/packages/core/agent_framework/security.py Outdated

Python: Scope FIDES session fallbacks by owner

1974a89

Oscar Lobato Ruiz (lerelerele) deployed to github-app-auth August 11, 2026 16:40 — with GitHub Actions Active

Merge branch 'microsoft:main' into codex/fides-7455-fixes

dafba0f

Oscar Lobato Ruiz (lerelerele) deployed to github-app-auth August 11, 2026 16:43 — with GitHub Actions Active

Oscar Lobato Ruiz (lerelerele) requested a balanced review from Copilot August 11, 2026 16:48

Copilot started reviewing on behalf of Oscar Lobato Ruiz (lerelerele) August 11, 2026 16:49 View session

Copilot AI reviewed Aug 11, 2026

View reviewed changes

Navigation Menu

Uh oh!

Python: fix FIDES session isolation and runtime integration gaps - #7528

Python: fix FIDES session isolation and runtime integration gaps#7528
Oscar Lobato Ruiz (lerelerele) wants to merge 9 commits into
microsoft:mainfrom
lerelerele:codex/fides-7455-fixes

Oscar Lobato Ruiz (lerelerele) commented Aug 5, 2026 •

edited

Loading

Uh oh!

Oscar Lobato Ruiz (lerelerele) commented Aug 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Evan Mattson (moonbox3) commented Aug 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Oscar Lobato Ruiz (lerelerele) commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Oscar Lobato Ruiz (lerelerele) commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Additional API surface

Scope

Tests and verification

Uh oh!

Oscar Lobato Ruiz (lerelerele) commented Aug 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Evan Mattson (moonbox3) commented Aug 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Oscar Lobato Ruiz (lerelerele) commented Aug 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Oscar Lobato Ruiz (lerelerele) commented Aug 5, 2026 •

edited

Loading